Busting the Myth of Televised Crime Dramas

Authentic crime scene investigating is far from an “exact science.” We are all victims of television crime dramas. And, unfortunately, so is the criminal justice system. At the beginning of 2017, I was asked to help my professor, Robert Sanger, research and draft legal standards for the American Academy of Forensic Sciences–Firearms and Toolmarks Committee. I was surprised to discover there are no uniform, national standards by which experts testify. I was further surprised to find that the “exact science” of firearms and toolmarks examination and identification was not really “exact”.

I began my quest by compiling all the information I could find from online and other sources regarding the topic. I was looking for commonalities among standards, perspectives on current and future standards, and court rulings on expert witness testimony. After compiling significantly more information than I could analyze in the time allotted, I realized there was legitimate and significant disagreement about what the standards should be, or even if consistent standards were possible.

Forensic Science Carries Inherent Uncertainties

The basis for expert witness testimony regarding firearms and toolmarks is that incidental scrapes and scratches in different materials (whether that be striations from a bullet passing down a barrel, or scratches on a door frame) are sufficiently unique, and that a trained forensic scientist can match those marks to other known or unknown evidence. Typically, this information has been presented as a “match” or “scientific certainty.” Unfortunately, because the assessment of the evidence is ultimately predicated on the scientist’s judgement, there is an inevitable uncertainty inherent with every determination.

Recently, these issues were highlighted in two reports: the President’s Council of Advisors on Science and Technology Report (PCAST) 2016 and the National Research Council’s (NRC) “Strengthening Forensic Science in the United States: A Path Forward” (2009). The PCAST report specifically states “concerns about certain foundational documents underlying the scientific discipline of firearm and tool mark examination.” It goes on to quote a 2008 NRC report finding that “the validity of the fundamental assumptions of uniqueness and reproducibility of firearms-related toolmarks’ had not yet been demonstrated and that, given current comparison methods, a database search would likely return too large a subset of candidate matches to be practically useful for investigative purposes.” This follows the 2009 NRC “A Path Forward”, which stated that “because not enough is known about the variabilities among individual tools and guns, we are not able to specify how many points of similarity are necessary for a given level of confidence in the result. Sufficient studies have not been done to understand the reliability and repeatability of the methods. The committee agrees that class characteristics are helpful in narrowing the pool of tools that may have left a distinctive mark.” In other words, what we think we know about our ability to match bullet casings to a particular gun is far from correct in most cases.

“CSI: Crime Scene Investigation” Isn’t Reality

This is a stark revelation to those raised watching crime dramas that show crime scene investigators (CSI) recovering a bullet and then running the striations on its side through a database until they find “a match.” It is from this “match” they are able to identify the murder weapon and, ultimately, the suspect. The reality is, high quality control at most modern firearms manufacturers means each firearm coming off the line is nearly identical to all of the others produced. A bullet fired from a particular model of firearm is nearly identical to a bullet from any other firearm of the same model; in some cases, it may be nearly identical to bullets fired from numerous other models. As a result, it is nearly impossible for an expert, no matter how diligent, to definitively state that a bullet recovered at a scene was fired by any particular firearm with certainty.

This is a serious issue, given that juries are comprised of people who may have been conditioned to believe that such a match is possible. Additionally, there is a culture within the forensic science community that also believes such determinations are possible. Unfortunately, without national standards, proficiency testing, and double-blind studies, that assessment lacks academic rigor. As a result, expert testimony can give juries a false sense of certainty as to the veracity of their judgements, and innocent people may be wrongfully convicted.

In order to ensure that the innocent are not wrongfully convicted, we must be willing to see the guilty go free. We must remind ourselves, and the community at large, that television shows are fiction and may overstate the capabilities of both the legal profession and law enforcement. Testifying experts in these communities must be willing to trust juries with the whole truth. And as members of these communities, we must remember that our commitment must always be to the truth.